kourouklidesfandomcom-20200213-history
Artificial Neural Network
This page contains resources about Artificial Neural Networks. Subfields and Concepts * Feedforward Neural Network ** Single-Layer Perceptron ** Multi-Layer Perceptron (MLP) ** Convolutional Neural Network (CNN or ConvNet) * Recurrent Neural Network (RNN) ** Hopfield Network ** Boltzmann Machine ** Bidirectional associative memory (BAM) ** Long short-term memory (LSTM) ** Continuous Time RNN (CTRNN) ** RNN-RBM ** Echo State Network (ESN) * Stochastic Neural Network (i.e. with stochastic transfer function and units or stochastic weights) ** Helmholtz Machine ** Boltzmann Machine ** Restricted Boltzmann Machine (RBM) ** Generative Stochastic Network ** Generative Adversarial Network * Radial Basis Function (RBF) Network * Kohonen Network / Self-organizing map (SOM) / Self-organising feature map (SOFM) * Bayesian Neural Network * Random Neural Network * Autoencoder (used for Dimensionality Reduction) ** Linear Autoencoder (equivalent to PCA) ** Stacked Denoising Autoencoder ** Sparse Autoencoder ** Generalized Denoising Autoencoder ** Variational Autoencoder * Deep Neural Network ** Deep Multi-Layer Perceptron ** Deep Belief Network (DBN) ** Convolutional Deep Neural Network ** Long short-term memory (LSTM) ** Deep Autoencoder * Training: ** Automatic Differentiation *** Backpropagation Algorithm *** Backpropagation Through Time (for RNNs) *** Stochastic Backpropagation ** Optimization *** Gradient Descent *** Gradient Descent with Momentum *** Simulated Annealing *** Genetic Algorithms (for RNNs) * Activation Functions / Transfer Functions for deterministic units (must be differentiable) ** Logistic ** Rectifier (ReLU) ** Softmax * Cost Functions ** Least-Squares ** Cross-entropy ** Relative Entropy / KL Divergence * Energy-Based Model (EBM) ** Free energy (i.e. the contrastive term) ** Regularization term ** Loss functionals or Loss functions *** Energy Loss *** Generalized Perceptron Loss *** Generalized Margin Losses *** Negative Log-Likelihood Loss * Improve Generalization (to prevent overfitting) ** Early stopping ** Regularization *** L1-regularization *** L2-regularization *** Max norm constraints ** Dropout Online Courses Video Lectures *Neural Networks for Machine Learning by Geoffrey Hinton - Coursera *Neural networks class by Hugo Larochelle (Youtube ) *Deep Learning and Neural Networks by Kevin Duh Lecture Notes *Convolutional Neural Networks for Visual Recognition by Fei-Fei Li & Andrej Karpathy Books and Book Chapters * Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press. * Theodoridis, S. (2015). "Chapter 18: Neural Networks and Deep Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press. * Barber, D. (2012). "Chapter 26: Distributed Computation". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Alpaydin, E. (2010). "Chapter 11: Multilayer Perceptrons". Introduction to Machine Learning. MIT Press. * Haykin, S. S., Haykin, S. S., Haykin, S. S., & Haykin, S. S. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson. * Bishop, C. M. (2006). "Chapter 5: Neural Networks". Pattern Recognition and Machine Learning. Springer. * MacKay, D. J. (2003). "Chapter 38: Introduction to Neural Networks" Information Theory, Inference and Learning Algorithms. Cambridge University Press. * Mandic, D. P., & Chambers, J. (2001). Recurrent neural networks for prediction: learning algorithms, architectures and stability. John Wiley & Sons. Scholarly Articles Tutorials Software See Deep Learning Software. See also * Probabilistic Graphical Models * Online Learning Other Resources * What is the difference between CNNs, RBMs and Autoencoders - Cross Validated Stackexchange * Neural Networks and Deep Learning - free online book * Neural Networks, Manifolds and Topology - Blog post Category:Machine Learning